The display of a three dimensional world to a viewer requires considerable computation power, and it is typically costly to develop the necessary highly detailed models required for doing so. In order to simplify the problem, a portion of the world that is in the distance may be represented as a video displayed on a surface, e.g., a screen. By video it is meant the common usage of the term, such as the placing or projecting of predefined images on the surface, e.g., the electronic version of filmed moving pictures. Thus, such a world is essentially truncated in length to the screen on which the video is displayed. A great reduction in computation power and cost can be achieved by such an arrangement.
A limitation of such a world occurs when an object represented by computer graphics undergoes a trajectory that takes it to a location in the world that is not represented as computer graphics but instead is within the field represented by the video. If the object is generated at its proper location using computer graphic techniques, without further processing such an object becomes invisible to a viewer in front of the video screen, because the object is blocked from a viewer's view by the video screen. In U.S. patent application Ser. No. 08/906215 it was recognized that, when a computer graphic object undergoes a trajectory that takes it to a location in the world that is not represented as computer graphics, but instead is within the field represented by the video, that such an object should be represented as video on the screen, rather than computer graphics. Doing so prevents the object from becoming invisible as it would if it was generated at its proper location using computer graphic techniques, since it would be blocked from view by the video screen. Thus, the computer graphics object "goes into the video" as video and remains visible to a viewer in front of the video screen.